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1 .  Introduction 

Consider  a  general  factorial  experiment  with  the  design  con¬ 
sisting  of  t  treatments  and  corresponding  to  the  uth  treatment  there 

t 

are  n  (>  1)  observations  and  E  n  =  N.  Let  y  be  the  observation 
u  —  u  uv 

u=  1 

corresponding  to  the  uth  replication  of  the  vth  treatment  and  be 
the  mean  of  all  observations  corresponding  to  the  uth  treatment.  The 
model  for  this  experiment  is 

E(y.)  -  X,  +  x2  Bj, 

V(y)  =  o2I,  (1) 

Rank  Xj  *  Vj  , 

where  is  a  vector  of  specified  lower  order  interactions  and 

8^(v2xl)  is  a  vector  of  some  or  all  of  the  higher  order  interactions, 
Xi(Nxv1)  and  X2(Nxv2)  are  known  matrices.  It  is  known  that  K  (very 
small  compared  to  v2)  elements  of  are  nonzero  and  the  other  are 
zero;  however  the  value  of  K  and  the  nonzero  elements  of  are 
unknown.  The  problem  is  to  search  the  nonzero  elements  of  $2  and 
draw  inferences  on  them  In  addition  to  the  elements  of  .  Such 
a  model  is  called  the  search  linear  model  and  was  introduced  in 
Srivastava  (1975).  Suppose  Kj  Is  an  initial  guess  on  K.  Note  the 
three  possibilities  Kj  >  K,  Kj  =  K  and  Kj  <  k.  We  consider  (^2) 


models 


E(y)  *  X,  B,  +  X^.  i-1 
V(jy)  *  o2I 

Rank [Xj ,X2*  ^  ]  *  vi  +  Kj  , 

AIR  ypc  —  ' 


(2) 


..  .  nnmiCmtS3caime> 
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where  xj^^NxKj)  is  a  subraatrix  of  X£  and  jB^^Kjxl)  is  a  subvector 

of  p 2*  It  can  be  seen  from  Srivastava  (1975)  that  we  in  fact  need 

Rank  [x4,  X^lj,  X^1^]  =  (v4  +  2Kt),  for  all  i  *  i’.  This  implies 

that  N  ]>  (v^  +  2K|).  In  case  K|  =  K,  one  of  (V2)  models  is  the 

correct  model.  If  Kj  >  K,  then  f^2  models  out  of  (v2)  models 

J\|  “K  lv| 

include  the  true  model  as  a  submodel  in  the  expectation  forms  of  the 


models.  The  methods  discussed  in  this  paper  will  not  only  identify  K 
nonzero  parameters  but  also  find  how  many  of  them  have  significant 
effects  and,  finally,  rank  the  significant  nonegligible  parameters  in 
the  order  of  their  influence  on  the  fitted  values.  In  case  Kj  <  K, 
the  methods  will  identify  from  Kj  parameters  the  parameters  which  are 
significant  and  influential.  We  also  propose  an  estimator  of  K  in  the 
Section  3. 

In  some  industrial  experiments,  it  is  often  easy  to  find  replica- 
cations  (n^  !>  1)  in  observations  corresponding  to  a  particular  (the 
uth)  treatments,  see  Taguchi  and  Wu  (1985).  There  are  also  situations 
in  industrial  experiments  where  it  is  impossible  to  get  replication  in 
observations  for  a  treatment,  see  Daniel  (1976)  and  Box  and  Meyer 
(1985).  The  methods  discussed  in  this  paper  consider  both  situations. 
In  all  Taguchi  design  methods,  the  higher  order  interactions  (2-factor 
and  higher  order  in  most  plans)  are  assumed  to  be  zero.  A  few  of 
those  higher  order  interactions  may  be  nonnegl igible ,  significant  and 
influential.  The  use  of  the  search  linear  models  may  be  a  potential 
tool  in  improving  upon  the  Taguchi  design  methods. 


u/l  .  I 
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2.  Influential  Konnegl i glble  Parameters. 

Let  zj15  ((K-v^-Kj)  x  N)  be  such  that  Rank  zj1*  =  (N-Vj-Kj), 
Z^zJ1*'  -  I  and  zj1^[x1,  X^1}  ]  =  0.  Let  Z(1)(K1  x  N)  be  such  that 

’z(1)  1 

J  i)  J  *  (N-va  ) ,  Z(1)Z(1)'  -  I,  Z(1)z5l5'»  0  and  Z(i)Xl  =•  0. 
It  can  be  seen  that  under  the  ith  model  in  (2),  the  minimum  variance 
unbiased  estimator  (MVUE)  of  is 


Rank 


‘(i>  -  (Z*1’*'1*)'1  -w~ 


h. 


z  y. 


(3) 


In  fact  we  can  write  Z 


(i)1 


where  is  a  nonsingular 


PjX^'D 


(and  triangular)  matrix  so  that  Z^^Z^^  =»  I  and  P| 


I  -  Xi(X^Xi)  ^x\ .  From  the  ith  model  in  (2),  the  MVUE  for  C|  is 


ll(1)  -  (xiXiTV^  -  (xixi)"1x'1xi1)|^) 


(4) 


The  fitted  value  of  y_  from  the  ith  model  in  (2)  is 

£(1)  *  XjI^15  +  Xg1^1} 

The  residuals  from  the  ith  model  in  (2)  are 

5(1>  -  z  -  i(1)  -  i’i(i-4,>lz1>) 


(5) 


-1 


pilI*X2i)(X21>  PlX2i))  * 


(6) 


The  sum  of  squares  due  to  error  under  the  ith  (i  -  l.»*»f(y2)) 


model  In  (2)  is 


SSE^  -  R<4)V*>  -  y’Z^V^y. 


(7) 
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The  residuals  under  the  model  (1),  when  6^  =  £»  are 


»<«>  ,  ri<°>  .  P,y. 


It  can  be  seen  that 


p  .(D'd)  .  7(D’7(i) 

Pi  *  Z|  +  z  z  . 


Therefore,  for  1  * 


se'o*  -  *<•»'«<<>>  -  sse“>  ♦  ,'E(1)'z(,)y. 


For  1  ■  we  define 

K1 


Z-*(1,,*(1V*. 


SSE^i^7(N-v1-KJ) 


Let  be  the  fitted  value  of  the  observation  corresponding  to  the 

u 


uth  (u  *  1 . w)  treatment  under  the  1th  model  In  (2).  We  write  the 


sum  of  squares  due  to  lack  of  fit  as 


SSLOF 


w  2 

I  a  (y  -y(1))  . 

U  v'u  'M  • 


and  the  sum  of  squares  due  to  pure  error  as 


v  a 

sspe  -  l  Iu  (y  -y  )  . 

wuv  u 

u»l  V*1 


For  1  *  we  defin 


,(i)  SSLOF^Vcv-Vj-Kj) 


SSPE/ (N-w) 
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Theorem  1.  For  t  c  { 1 .  the  following  statements  are 

equivalent. 

(O 

(a)  SSE  is  a  minimum, 

(l) 

(b)  F'  is  a  maximum, 

(t ) 

(c)  SSLOF  is  a  minimum, 

(O 

(d)  F  '  is  a  minimum, 

LOF 

(e)  The  Euclidean  distance  between  and  is  a  maximum, 

(f)  The  square  of  the  (sample)  simple  correlation  coefficient  between 

the  elements  of  R(l)  and  R(o)  is  a  minimum. 

Proof.  V*e  have  from  (10)  and  (11)  that 


F(i)  I  +1 


SSE 


(o) 


^(N-Vj-Kj)  J  SSE^1^ 

Noting  that  the  numerator  on  the  RHS  of  the  above  expression  does  not 
depend  on  i,  we  get  the  equivalence  of  (a)  and  (b).  Again, 

SSE^*  =  SSPE  +  SSLOF^, 

and  SSPE  does  not  depend  on  i.  Therefore  (a)  and  (c)  are  equivalent. 
From  (14),  the  equivalence  of  (c)  and  (d)  is  cleaT.  Ftotb  (3),  (6), 
(8)  and  (9),  It  follow  that 

rzU>V‘>2  -  |<1)V;1>Yi,Y1M,)  4° 

-I<1)’x<i>'[r(Y11>Y1‘>]  x<‘>|<‘> 

-  |‘1>'x<i>'F141)5'1)  US) 

-  (41,-4°))'  (sh)-r(o)) 

.  (-£<».£<-»).  (-£<1)+£<o)). 


The  equivalence  of  (a)  and  (e)  Is  now  easy  to  see  from  (10)  and  (15). 
It  follows  from  (10)  and  (15)  that  R^R^  =  R^'r^0*.  We  thus 


*(1>V4> 


m 


-  the  square  of  the  (sample)  Simple  correla¬ 
tion  Coefficient  between  and  fS°\ 

The  equivalence  of  (a)  and  (f)  Is  now  clear  from  (16).  This  completes 
the  proof  of  the  theorem. 

Propost ion  1.  Under  the  1th  model  in  (2), 

_(i)  (i) 

z  R  -  o.  (17) 

?SP?L'  It:  follows  from  (3)  and  (5)  that 

*<0z  -  zt,)xii)£1)  - 

This  completes  the  proof. 

We  have 

v(£(1))  -  [x-x51>(x‘»,r1x^>)',xJ^P1.  <i8) 

The  residual  In  R^  ^  are  correlated  and  the  question  may  be  asked 
about  the  appropriateness  In  combining  the  elements  of  R^  in  SSE^. 
If  we  take  the  transformed  residuals  as  Z^R^,  we  then  have 


(R(1)V0))2 


E(Z(11}R(1))  -  O  and  V^V1*)  -  o2l. 
The  sum  of  squares  of  these  transformed  residuals  Is 

R(i)'x1(i),21<1)R(i). 


V.vl»irV3 


o  o  or*  1 •  *.v< 
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Proposition  2.  For  i  * 

SSE<‘>  - 

Proof .  We  write  the  RHS  using  (9)  as 

i)  _  ,<i)  »(i),»(i),za)R<o 


2i  5. 


P,R  -  R 


It  can  be  checked  that  using  the  Proposition  1-,  the 

rest  of  the  proof  is  clear.  This  completes  the  proof. 

Proposition  2  thus  supports  the  use  of  SSE^^.  Theorem  1  gives 
various  interpretations  of  a  search  procedure,  discussed  in  Srivastava 
(1975),  of  selecting  as  the  influential  set  of  K|  nonnegl igibl e 

parameters . 


We  now  denote 


1(1)’  .  fV1*  8(i>  6(i>l 

*2  P21  .  2KJ* 

,<*>  .  rx<i)  x(l)  x(i)l 

2  [-21  . -2j  -2KJ- 


4*1  > 


the  matrix  obtained  from  X%  by 


deleting  the  jth  column  of  X^j  * 

x<|3>  -  [xJt  *<«>]. 

p<iJ)  -  i-4P)(x^),4l3)r14P)’. 


12  -  2j 


MO’  (iJJyCi) 

— 2j  *12  — 2j 


.«>•  .r.(i)  z<*>  Z<*>1 

Z0  [in  — ij  -1K|J* 
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I t  can  be  seen  that 


Rank  (i)'  =(N-vj-Kj+1)> 

4 £•  _  # 


7(i)’  (i)  .  ,  z^)'z(i)_0. 

Z1<  1,4  ~  *»  1,4  Zi  *  2  » 


“Ij 


j^z(i)’J  X^j)  -  0,  Rank  zj0  -  ^ . 

There  exists  a  nonsingular  (triangular)  matrix  such  that 

Z^  =  Dq1^  Z^1^ . 

From  (3)  and  (24),  we  have 

U1’  -  (4i)41,)-,41,y- 


7<1,y<1)  -  t,M\W  -(O'  u>  z(1)'x(°l  (26) 

z0  X2  “  diaS  '-11  X21  »***»Zij  X2j  . -IK  -2K  J 


is  a  diagonal  matrix.  Thus 

g<4>  __Z  -  .  0 

2J  _C±>*  CD 

“lj  “2j 

Let  R(1^,  i  =  1,...,  (y2),  j  -  be  the  residuals  obtained 

Ki 

from  ith  model  in  (2)  assuming  =  Then  the  sum  of  squares 

due  to  error  is 

SSE^1-^  =  =  (Z^’y)2  +  SSE^ 1  ^ .  C 

We  now  define,  for  i  =  I,...,^2)  and  j  * 


VT 
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t(ij) 


7(i)’ 

*1J  * 


SSE*'1^ 

(N-v7-k7) 


(29) 


Proposmop^.  For  a  fixed  £  In  {l,...,(^)]  and  an  m  In  {l . Kj } , 

the  following  statements  are  equivalent. 

(a)  SSE  is  a  minimum, 

/v\  .  m)  . 

i  b)  t  is  a  maximum# 


Proof.  The  proof  can  be  easily  seen  from  (28)  and  (29). 

(  £) 

In  the  set  8^  of  influential  nonnegl igible  parameters,  8^^  is  the 
most  influential  nonnegligible  parameters.  The  influential  non- 
negligible  parameters  may  or  may  not  have  significant  effects  on 
observations. 


.  •*- 


A 


V*Va*.'V..;.^Lr 


K* 


■inrun HJ*  'Ml  ' CT  'J  rv* 


y-y w  Kn i^np*  m  w-«r  w  p  »  g  *  w  *  v n  u ■»  w  w u  n  u  ■  vrw  w 


J\V 
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3.  influential _  Significant  Nonnegliglble  Parameters 

We  now  assume  the  normality  in  (2)  and  therefore  for 

1  -  1 . (£).  Hfx.e.  +  X21)?21),  «’l>. 

Under  the  null  hypothesis  Hq  :  Bj*^  ’  0,  hss  the  rentral  F  dlslrl- 

button  with  (Kj  ,  N  -  -  K^)  d.f.  and  under  the  null  hypothesis 

V  ^  =  t^  ^  has  the  central  t  distribution  with  (N  -  -  Kj) 

d.f.  We  now  present  a  further  development  of  a  procedure  suggested  in 
Srivastava  (1975). 


Case  _I.  If  max  ^ 

i 

is  no  significant  nonnegliglble  parameter.  (F 


<  FQ_^  ^  v  ^  ,  we  then  conclude  that  there 


ajK^N-Vj-K, 

upper  a  percent  point  of  the  central  F  distribution  with 


is  the 


(Ki  .N-Vj-Kj )d . f . ) . 


(i) 


Case  II.  Suppose  for  i  =  i  ,  we  have  F  >  F  „ 

•  1  S  a;Kj  ,.%-Vj-Kj 

We  denote  for  j  =  l,...,v2, 

3.  =  the  number  of  i  in  {i  ,...,i  }  for  which  It  ,1  >  t 
J  Is  I  ljl  £  N-Vj-Kj  . 

2 

Note  that  0  <  3^.  <  s.  We  now  arrange  S^’s  in  decreasing  order  of  mag¬ 
nitude  and  write  3,,.  >  3.  .  >  ...  >  3,  .  .  If  there  are  at  least  K. 

(1)—  (2)-  —  ( v2 )  1 

nonzero  3^  j's,  we  select  the  influential  significant  parameters  as 
8( i ),..., 8(i^  otherwise  we  pick  the  influential  B^^'s  corresponding 
to  nonzero  3^^'s  that  the  number  of  Influential  parameters  is 

then  less  than  K| ) .  The  parameter  B^^  is  n>ost  influential 
significant  nonnegliglble  parameter.  An  estimator  of  the  unknown  K  is 

'N 

K  =  the  number  of  nonzero  3  'S.i  =  ],..., v  . 

j  2 


Miscellaneous  Results 


Let  us  denote  the  unknown  nonzero  elements  of  *n  CO  by 

8  (Kxl)  and  the  zero  elements  of  8  by  J3  ((v  -K)Xl),  *be  cor~ 
2  c  —2  * 


-2d' 

responding  columns  In  X  matrix  are  X„  and  X„. 

2  2c  2d 

true  expectation  form  of  (1)  is  thus 


-  *,1,  +  *2cV 


The  unknown 


(30) 


The  expectation  form  of  the  ith  model  in  (2)  can  be  written  as 

My)  -  V,  +  (31) 

where  X^^CNxy^)  is  a  subnatrix  of  X ^ [Nx( Kj )  is  a 

submatrix  of  X^,  jJ^^y^xl)  ls  a  sut,vector  of  and  ®2d^ 

((Kj-y^Xl)  is  a  subvector  of  JJ .  Let  ls  vector 

of  elements  in  8_  which  are  not  in  and  X*^^  is  the 

—2c  —2c  2c 

matrix  whose  columns  are  in  X_  but  not  in  The  following 

2c  2c 

result,  a  counterpart  of  the  result  in  (10)  for  the  population ^ 
can  be  verified  very  easily. 

Proposition  4.  Under  (30), 


E(SSE(1))  *  E(SSL0F^1;)  +  oZ(N-v) 


(1) 


■  (dot-Vj-ic,)  +ijcxjcz(1i),z(11)x2<:62c 


(1)'  (1)'  (i)«  (1)  (i)  (i)  (32) 

v*  7'  '  •?  V*  04- 


X*2c  —2c 


o  (N-Vj-Kj )  +  B$c  X^c  Z\  Zj 
E(SSE(0))  -  [o2Kj  +  I^X^cZ(i),Z(i)X2e62c]  . 


r .  / 
ML  VV 


4.b.  The  model  obtained  from  (2) 


E(Z(1)y)  = 
V(z(0y)  =  o2I, 


(i)v(i)fl(i) 


Is  called  the  pure  search  model  (Srlvastava  (  1976)).  Tn  fact, 
Srivastava  (1976)  considered  a  special  form  of 
4.c.  The  Influential  nonnegllgible  parameter  may  depend  on  noise, 
i.e.,  a  parameter  may  be  influential  under  one  noise  but  may 
not  be  influential  under  another  noise. 

4.d.  The  replicated  observations  will  surely  improve  the  chances  of 
detecting  the  correct  influential  nonnegligible  parameters. 

4.e.  Iu  presence  of  outliers  in  observations,  one  may  combine  . 
residuals  with  unequal  weights,  or  in  other  words,  may  use 
transformed  residuals  (see.  Cook  and  Weisberg  (1982)). 

4.e.l.  An  example  of  transformed  residuals  is  the  vector 

where  M^^(NxN)  is  a  diagonal  matrix  whose  uth 

•  _ _ \  /j  \ 

diagonal  element  is/  1/  /  (i)  with  mv  being  the  uth 

\  J  m  uu 

'  v  uu 


_2  (  i  ) 

diagonal  element  of  0  v(]R  ). 

4.e.2.  Suppose  the  underlying  design  is  robust  against  the 
unavailability  of  any  single  observation  [see,  Ghosh 
(1980)]  in  the  sense  that  the  estimation  of  and 
is  possible  under  (2)  when  any  single  observation  is 
unavailable.  We  find  the  predicted  value  of  the  Uth 
observation  from  the  remaining  (N-l)  observations  (i.e. 


by  deleting  the  lith  observation).  The  difference  between 
the  uth  observation  and  its  predicted  value  is  called  the 
uth  predicted  residual  (using  the  idea  of  cross  valida¬ 
tion).  It  can  be  verified  algebraically  that  the  vector 
of  predicted  residuals  is  [m^^]  The  predicted 

residual  sum  of  squares  (PRESS)  from  the  ith  model  under 
(2)  is 

PRESS^  =* 

In  presence  of  outliers,  one  may  take  PRESS^1^  as  an 

(i) 


alternative  to  SSE 
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